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Abstract 


Selecting and remembering the necessary information from rapidly increasing information requires sum- 
marization skills. Research on improving students’ summarization skills has focused on deletion, general- 
ization, and reconstruction strategies. However, direct teaching of these strategies does not yield success- 
ful results. For this reason, researchers have turned to teaching summarization based on understanding 
the text. Knowing the macrostructure and superstructure of the text can help understand the text and 
select the information to be included in the summary. The aim of this study is to determine the effect of 
macrostructure and superstructure teaching on summary writing achievement. The study was conducted 
in a one-group pretest-posttest design and lasted for 6 weeks. The study was conducted with 22 seventh 
grade students in a public secondary school. The data were collected with a total of 132 summary texts 
in which students summarized five stories. The summaries of texts were scored with the Text Summary 
Evaluation Rubric (TSER). As a result of the study, significant differences were obtained in summariza- 
tion achievement in favor of the posttest. Based on this result, it is recommended that macrostructure and 
superstructure teaching be included in programs and course books. This study is limited to narrative 
texts. In future studies, whether the achievement in summarization narrative and informative texts differs 
and how long it takes for different age groups to develop their achievement in summarization of different 
types of texts can be examined based on text structure teaching. 

Keywords: Macrostructure teaching, superstructure teaching, summarization achievement, seventh 
grade students 


Introduction 


Innovations in the world of information and technology and the ability to spread 
information across the world in seconds have made it easier to access information. However, 
understanding the information presented, selecting the important ones, and remembering 
them when necessary have become important skills in order to benefit from the rapidly 
increasing accumulation of knowledge. An effective way of developing these skills is teaching 
summarization. Summarization is the “reductive transformation of the source text into a 
summary text by condensing the content through selection and/or generalization about what is 
important in the source." (Sparck Jones, 2007, p. 6). Summarization is a multi-layered skill that 
involves both comprehension and expression skills. For this reason, it should not be considered 
as a skill that students can acquire without training. 

While teaching summarization, students should be explained what they should do before, 
during and after summarization and guidance should be provided to support their learning. 
The basic stages of summarization are comprehending the text, selecting or creating important 
information, shortening the text without losing the main message, and writing fluently in their 
own words. In this respect, summarization is primarily based on reading comprehension. 
There is a reciprocal relationship between summarization and reading comprehension. Good 
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readers write good summaries (Bahap Kudret & Baydik, 2016; Pirc & Pecjak, 2018), teaching 
summarization strategies increases reading comprehension (Belet, 2005; Dollins, 2012; Graham 
& Hebert 2010; Khathayut & Karavi, 2011; Khoshsima & Rezaeian Tiyar, 2014; Nelson, Simith 
& Dodd, 1992; Nurhayati & Fitriana, 2018; Pakzadian & Rasekh, 2012; Sucita & Hukom, 
2022). Bahap Kudret and Baydik (2016) found a positive, moderate relationship between 
reading and summarization achievement. However, reading comprehension and understanding 
the text for summarization are not the same. Writing summaries is not a natural consequence of 
understanding and remembering the text; it is necessary to use deliberate processing strategies 
for summarization as well as comprehension (Hidi & Anderson, 1986). The process of selecting 
particularly important propositions and reducing them, in which propositions are deliberately 
condensed through various higher-order transformations, is a key feature of reading for 
summarization (Winograd, 1984 as cited in Hidi & Anderson, 1986). 

In the process of teaching summarization, first of all, analyzing good and bad summaries 
and starting with the completion of incomplete summary texts will provide a concrete view of 
what can be included in the summary (Blanchard & Root, 2004). After the summary structure 
is introduced, summarization strategies can be taught. The deletion, generalization, selection 
and construction strategies developed by Kintsch and van Dijk (1978) are frequently used in 
teaching summarization strategies. According to Kintsch and van Dijk (1978), summarization is 
the technique of creating the macrostructure of the text by identifying its important points. The 
macrostructure is a structure that preserves the general meaning and structure of the original text 
but is stripped of details. According to this model, each sentence that forms the basis of the text 
and the relationship between sentences constitutes the microstructure. Deletion, generalization, 
and construction operations are applied to create a macrostructure from a microstructure. In 
the deletion process, a sentence is deleted if it is insufficient to represent the whole and creates 
a coincidental connection with other sentences. In generalization, similar concepts related to 
the same concept area in the text are generalized. In structuring, the propositions in the text are 
evaluated and transferred to a new set of propositions. 

Deletion, generalization, and construction are the ultimate goals of summarization. 
However, research shows that teaching summarization based only on these strategies does not 
produce very successful results. Students have difficulty in selecting important information 
(Ozcakmak, 2015; Friend, 2001), deleting unimportant information (Garner, 1982) or 
construction (Kim, 2001). In the experimental study conducted by Aydin (2022), secondary 
school students were taught deletion, generalization, and construction strategies. As a result of 
the study, it was determined that students did not show improvement in any of these strategies. 
Rather than teaching "deletion, generalization and reconstruction" strategies directly, the 
researchers focused on text comprehension studies for summarization. In summarization 
teaching, it is tried to improve the ability to select important information by using text-oriented 
questions (Boga, 2019), mind maps (Aksoy, 2022; Aulia, 2017), story maps (Arslan, 2017; 
Sahin, 2012) or thinking about the text, associating ideas with each other (Friend, 2001). The 
common feature of these studies is content-based summarization. Content-based summarization 
is a summarization technique based on the main content of the text (Cahyono, 1996). This 
“summarization technique is able to improve student’s comprehension and their summaries 
skill especially in cases of finding the main ideas, committing plagiarism and committing 
distortion” (Khathayut & Karavi, 2011, p. 7). 

The results obtained from the literature review have shown that it is important to develop 
comprehension for summarization purposes in summarization teaching. In order to achieve this, 
this study adopted a summarization teaching process supported by the teaching of strategies for 
identifying the macrostructure and superstructure that make up the text. The macrostructure is 
a structure that takes the text as a whole, reflects the general meaning of the text and constitutes 
the content of the text. Without creating the macrostructure of the text, it is impossible to 
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make semantic organization by eliminating complex information and to perceive the general 
meaning of the text (van Dijk & Kintsch, 1983). Working on macrostructure is important for 
both understanding the text and determining the elements to be summarized. This is because in 
macrostructure, the main topic is the most basic structure, and determining the main topic of a 
text and the subtopics that lead to this topic is essential for understanding the text. The elements 
of macrostructure that ensure the coherence of the text in terms of its topic include the title, 
topic sentence, keywords, main and auxiliary thoughts/events, content schema, topic change 
determinants and a conclusion sentence. 

The type of text determines which information should be included in the summary text 
(Dilidiizgiin, 2013, p. 51). For this reason, it is important to conduct summarization teaching 
based on text types. The genre-specific structuring of the text is the superstructure of the text. 
The superstructure of narratives consists of "setting, initiating event, internal response, attempt, 
consequence, and reaction” (Stein & Glenn, 1979 as cited in Chen & Su, 2012, p. 186). The 
setting element includes time and place; initiating the event is the main problem, attempt is the 
main character's first reaction to the main problem, consequence is whether the main character 
solves the problem or not, and the reaction is the main character’s response to the consequence 
(Chen & Su, 2012). All of these elements should be included in the summary of a narrative text. 
Recognizing the genre-specific structure of narrative texts is useful for determining what should 
be included in the summary. 

The next stage in summarization teaching is writing the summary. At this stage, writing 
skills come into play. Writing an original text is based on planning the basic ideas and details, 
but summary writing is based on comprehension, evaluating, condensing, and transforming 
the original text (Hidi & Anderson, 1986). When writing a summary, it is possible to sort the 
selected or created sentences according to the subject/event order, to make them into a coherent 
whole by using transitional link expressions, to write with one's own sentences, to put a new 
title, to ensure time coherence and to check the consistency between the summary and the 
source text. 

Teaching summarization is a part of foreign language teaching and mother tongue 
education. In the Turkish Language Teaching Programme, summarization is included both as 
a reading and writing strategy and as a learning outcome. However, deletion, generalization 
and construction strategies are not included in the curriculum and course books. Students are 
asked to write a summary within the framework of the instruction "summarize the text" in the 
course books. However, students are not shown how to do this, and they are not given regular 
feedback on their summaries (Aydin, 2022). Students' receiving feedback is an important factor 
in the development of summarization skills (Ozdemir, 2018). As a result of the lack of regular 
teaching, summarization problems are seen at all levels from primary school to university. 
Students fail to distinguish important information from unimportant information (Aktas & 
Bayram, 2017; Aydin, 2022; Bahap Kudret & Baydik, 2016; Erdem, 2012; Karatay & Okur, 
2012; Ozcakmak, 2015), frequently include quotations in their summaries (Cikrik¢1, 2008), and 
do not use the tense of expression correctly (Sulak & Arslan, 2017). It is possible to transform 
unsuccessful summaries into ideal summaries by teaching summarization strategies. There are 
many studies confirming this claim (e.g. Aksoy, 2022; Anderson & Hidi, 1989; Arslan, 2017; 
Aulia, 2017; Boga, 2019; Cahyono, 1996; Friend, 2001; Ozdemir, 2018). 

Experimental studies on teaching summarization strategies are generally conducted with 
university students and are generally aimed at summarization informative texts in a foreign 
language (e.g. Aulia, 2017; Bahrami & Rahimy, 2022; Febriani et al., 2019; Friend, 2001; 
Khathayut & Karavi, 2011; Kim, 2001; Li, 2016). It is seen that experimental studies on the 
summarization of informative texts in the mother tongue were conducted at the primary school 
level (Armbruster et al., 1987; Arslan, 2017; Ozdil, 2019; Westby et al., 2010) and secondary 
school level (Aydin, 2022; Cikrikg¢1, 2008). Summarization teaching based on narrative text 
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structure teaching was conducted with primary school students (Hellmann & Ehri, 2020) and 
university students (Chen & Su, 2012). There is no experimental study focusing on both the 
macrostructure and the superstructure of narrative texts conducted in the mother tongue and with 
secondary school students. The study is important because it will fill this gap in the literature. 


Research Problem 


Identifying the elements that make up the macrostructure of the text can be a way 
of distinguishing important information from unimportant information in summarization. 
Macrostructure elements are presented in an order specific to the text type, that is, according 
to the superstructure of the text. Macrostructure and superstructure information can be guiding 
in determining what to include in the summary. In the literature review, it is seen that no 
summarization study has been conducted to teach strategies for these two structures together. 
Knowing these structures may reduce the problems encountered in summarization. 


Research Focus 


The focus of the research is to determine how knowing the macrostructure and 
superstructure of narrative texts affects secondary school students' summarization achievement. 


Research Aim and Research Questions 


This research aims to examine the effect of summarization teaching supported by teaching 
macrostructure and superstructure determination strategies on the development of secondary 
school students’ achievement in summarization narrative texts. The subject was addressed with 
two research questions: 


1. How is the weekly development of secondary school students’ achievement in 
summarization narrative texts after summarization teaching supported macrostructure and 
superstructure identification strategies? 


2. What is the effect of teaching macrostructure and superstructure on summarization 
natrative texts achievement? 


Research Methodology 
Research Design 


The research was conducted with one group pretest-posttest model which is one of the 
one group pre-test post-test designs. In this design, an independent variable is applied to a 
group. Measurements are made before and after the treatment. If the posttest scores are greater 
than the pretest scores, it is accepted to be from the independent variable (Karasar, 2009). The 
experimental process of the research lasted 6 weeks in the spring term of the 2021-2022 academic 
year. The study was conducted for 2 lesson hours per week. In the first week, pretest data were 
taken, and no training was given. In the second week, the process of teaching summarization 
strategies was started, and in the third and subsequent weeks, studies were continued for the 
strategies in which the students did not show improvement. The experimental process was 
completed when the summarization achievement exceeded the average in all criteria. The 
experimental process is explained in detail below. 
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Study Group 


The study group of the research consists of 7th-grade students studying in a public 
school. The experimental process was carried out with students taking the "Authorship and 
Writing Education" course in the spring term of the 2021-2022 academic year. The class size 
was 30, but the study was completed with 22 students due to 6 students who were absent some 
weeks and 2 students who did not want to participate in the study. 13 boys and 9 girls, a total 
of 22 students participated in the study voluntarily. Approval for the research was received 
from Bartin University Social and Human Sciences Ethics Committee. The ethics committee 
approval number is 2022-SBB-0034. 


Data Collection Tools 


The data of the study were collected with a total of 132 summary texts in which students 
summarized 5 stories. In determining the texts to be summarized, 10 stories suitable for the 
level of secondary school students, with easy and medium difficulty levels of readability 
were determined by the researchers. The determined stories were presented to two Turkish 
language teaching experts. The experts analyzed the texts in terms of the criteria of having clear 
macrostructure and superstructure features and being suitable for the level of students. The 5 
narrative texts that both experts found appropriate according to all criteria were used in the 
study. The medium difficulty story named Kirmizi Pabuc¢lar (Red Shoes) was used in the pretest 
and posttest. Information about the summarized stories is presented in Table 1 in the order of 
weekly study. 


Table 1 
Information about the Summarized Stories 


Week Text Number of waithor Readability 

Words Level 

1 Kirmizi Pabuclar (Red Shoes) (Pretest) 605 Nehir Tunaz Moderate 

2 Kaynatilmis Tohum (Boiled Seed) 752 Murat Ertan Easy 

3 Portakal (Orange) 813 Mustafa Ciftgi Easy 

4 Kasagi (Grooming Brush) 698 Omer Seyfettin Easy 

5 Son Kuslar (Last Birds) 576 S. Faik Abasiyanik Moderate 

6 Kirmizi Pabuclar (Red Shoes) (Posttest) 605 Nehir Tunaz Moderate 


Text Summary Evaluation Rubric 


The summaries of the above texts were scored with the Text Summary Evaluation Rubric 
(TSER). The scores obtained were used as the achievement score for the summarization of 
narrative texts. The rubric was developed by Benzer et al., (2016). The rubric includes 12 criteria 
related to "form, content and style" dimensions. The criterion of "side thoughts/events" in the 
content criteria of the rubric was excluded from the analysis because it covers the criterion of 
"plot". The criterion of "writing a new title", which 1s directly related to the content of summary 
texts, was added. The rubric is presented in Table 2. 
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12. Direct citation or 
imitation 


It consists of direct 
citation or imitations 
from the main text. 


sentences, which 
makes direct citations or 
imitations from the main 
text. 


Theme Criteria Inadequate (1p.) Need to Improve (2p.) Successful (3p.) 
2: Pacarorek Paper order is not Paper order is not Paper order is sufficient 
onee respected. respected partially. and placed in the center. 
euminaly sas Summary text consists of Summary text consists of 
2. Paragraph number consists of 7 or more AG caraaraohs 4-3: caracraphs 
SHAPE paragraphs. al isa ee ail 
6 and more grammar, 3-5 Grammar, 1-2 Grammar, 
3, Grammar, punctuation, and punctuation, and spelling punctuation, and spelling 
punctuation, spelling spelling mistakes have _— mistakes have been mistakes have been 
been made. made. made. 
4. Writing a new title — The title is not written. Th ils athe vied Mi Anew ills hiss Deen 
is written verbatim. added to the summary. 
5. Content integrity Content integrity is not Content integrity is not Content integrity is 
respected. respected partially. satisfied. 
6. Introduction There is no introduction Introduction sentence is The introduction sentence 
sentence sentence. inadequate. gives the topic of the text. 
In summary, topic and — In summary, the topic and In summary, the topic and 
7. Plot event order are not paid event order were messed _ event order are given in 
CONTENT attention to. up. the correct way. 
Uhpiatessaly Celalls Unnecessary details and —_ Details and unrelated 
. and unrelated ; ; : pats . 
8. Details ; : ._ unrelated information to —_ information with the topic 
information to the topic ; ; ea 
. the topic are given less. aren't given. 
are written. 
Keywords have not Soi ouiie eayweits All keywords have been 
9. Use of keywords ‘ have not been : 
been mentioned. : mentioned. 
mentioned. 
re The main idea has not Themainideahas been The main idea has been 
10. Main idea : : : ; 
been issued. issued partially. determined. 
(i liseettine Time suffixes have Some of the time suffixes Tiniecautiixes have Beat 
been used as have been used as : 
suffixes : : : ; used as compatible. 
incompatible. incompatible. 
STYLE It consists of fewer 


It is written with the 
expression of the reader. 


Procedure 


Week 1: No training on summarization strategies was applied. Students summarized the 
Kirmizi Pabuclar (Red Shoes) text based on their existing knowledge. These summaries were 
used as a pretest of summarization achievement. 

Week 2: Narrative text superstructure was introduced. Information was given about the 
macro scale structure elements such as topic sentence, main idea sentence, title, keywords, 
topic change determinants, conclusion sentence and content scheme. These were exemplified in 
a story they had previously studied in the textbook. Deletion, generalization, and construction 
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strategies were introduced, and a sample study was carried out on the same story. Based on the 
criteria in the rubric, a summary text structure was introduced. Successful and unsuccessful 
examples were selected from the summaries of the first week and shown on the smart board 
and the reasons why these examples were successful or unsuccessful were discussed. Then, 
the evaluations of the previous week's summaries with the rubric were given to the students. 
Finally, the students summarized the story of Kaynatilmis Tohum (Boiled Seed) within the 
framework of the information they learned. 

Week 3: The detailed evaluation of the Kaynatilmis Tohum text with the rubric was 
distributed to the students to see their mistakes. In addition, successful, average and unsuccessful 
examples were projected on the screen. In the first two weeks, it was observed that the most 
common problems of the students were not starting the summary with an introductory sentence 
reflecting the topic and not providing integrity of meaning, including details and direct 
quotations. Information was given on these issues and the story of the previous week was 
worked on. The students were asked to look at their summaries with a critical eye and share the 
mistakes they found. Finally, they were asked to summarize the story called Portakal (Orange). 

Week 4: Examples of successful, average, and unsuccessful summaries of the previous 
week were discussed. The evaluations made with the rubric were given to the students to see 
the strengths and weaknesses of their summaries. It was determined that the students could 
not show enough improvement in the criteria of ensuring the integrity of meaning, including 
the topic sentence in the introduction, using keywords, plot, and main idea. The narrative text 
structure, subject change determinants and the elements that make up the content scheme of the 
text were analyzed on the story of the previous week. In order to determine the keywords, it 
was explained that they should pay attention to the title of the text, the most frequently repeated 
words, the topic and main idea sentences, and the sentences expressing the changes in the 
protagonist's reactions, thus the changes of the main event. The keywords of the story titled 
"Portakal" were determined and how they could summarize the text with just keywords was 
illustrated. It was emphasized how using transition and connecting expressions when writing a 
summary ensures fluent expression. The students summarized the text called Kasagi (Grooming 
Brush) within the framework of the information they learned. 

Week 5: After examining the sample summaries of the previous week and distributing 
the rubrics to the students, studies were carried out on the subjects that the students could not 
improve. The problems seen in the previous week continued to be seen. The topics emphasized 
in the previous week were repeated in the story of Kasag1. Students summarized the Son Kuslar 
(Last Birds) story. 

Week 6: Examination of the previous week's sample summaries and evaluations made 
with rubrics were distributed to the students. In the summaries of the previous week, problems 
were identified in terms of grammar, punctuation, spelling mistakes, integrity of meaning and 
consistent use of tense suffixes. Work was carried out on student summaries to correct errors 
on these issues. In addition, the number of students who failed in the criteria of introductory 
sentence, plot, details, keywords, and main idea increased compared to the previous week. In 
these subjects, analyses were made on the story of the Son Kuslar (Last Birds). However, it was 
decided to complete the experimental process since the average score of 2 and above was seen 
in all criteria. The summary of the Kirmizi Pabuclar (Red Shoes) was written as the final test. 


Data Analysis 


In order to determine the weekly development of summarization skills, frequency analysis 
was performed using the measures in the rubric. Since the summarization achievement scores 
were not normally distributed, the Wilcoxon Signed Ranks Test, one of the nonparametric tests, 
was used to determine whether there was a significant difference between the pre and post test 
data. 
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Reliability 


The students' summaries collected as pretest and posttest were coded by two raters based 
on the criteria in the rubric. The agreement between the coding of the raters was analyzed by 
Cohen's Kappa Analysis. The kappa value is between -1 and +1. As the value approaches +1, 
the level of agreement between the raters increases and weakens as it approaches -1. Kappa 
value agreement scores are <.00 weak, .00-.20 insignificant, .21-.40 moderate, .41-.60 generally 
compatible, .61-.80 high and .81-1.00 very high (Landis & Koch, 1977). In this study, it was 
determined that the pretest scores were generally compatible with the Ist and 3rd criteria, highly 
compatible with the 2nd and 12th criteria and very highly compatible with the other criteria; 
the posttest scores were highly compatible with the 12th criterion and very highly compatible 
with all other criteria. 


Research Results 
Results Related to the Weekly Development of Students' Summarization Achievements 


To explore the weekly development of students' summarization achievements descriptive 
analysis was conducted. The analysis results are seen in Table 3. and Table 4. 


Table 3 
Weekly Development of Summarization Achievement 


Form Content Style 
B Fee é& Z gs 
=. ws Ese © §£ <« S &8 @ & 2 ne 
B £& 88 55 €222 8 2 32 
& £- 98 £ € 3 Ss 2» 5 F gf 
a: Se 8 § 
A a ~ = co a = 
G = 
= Inadequate 10 19 3 10 21 10 15 10 21 14 19 
3 Need tolmprove 11 1 10 12 1 10 7 12 1 1 
= Successful 1 11 2 9 0 0 2 0 0 0 1 2 
i Inadequate 3 2 7 0 10 15 7 10 7 9 12 10 
$ Need tolmprove 11 10 8 1 11 6 14 8 13 6 8 10 
a Successful 8 10 7 21 1 1 1 4 2 7 2 2 
oo Inadequate 2 5 10 16 #8 9 10 16 6 9 
$ Need to Improve 11 9 9 6 9 8 11 5 8 9 
a Successful 9 12 8 22 «3 es) 1 1 8 4 
& Inadequate 2 2 ri 0 5 2 3 3 4 4 5 
$ Need to Improve 7 6 ] 0 12 2 12 10 11 7 8 ] 
a Successful 1314 6 22.25 13 8 9Q 8 11 10 8 
1 Inadequate 7 0 4 5 4 98 3 6 
$ Need to Improve 5 1 8 8 11 8 8 7 
= Successful 11 11 10 21.4#10 12 #9 7 9 13 11 9 
“és Inadequate 2 0 3 0 2 4 2 5 
& 
& Need to Improve 5 1 7 5 5 1 4 5 
- Successful 1521 12 22 14 #18 #+15 #13 © «£13 ~=~=«19 15 12 
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When the summaries of the Ist week are analyzed, the most successful criteria are the 
number of paragraphs (f= 11) and putting a new title (f= 9). There are no successful students 
in the criteria of integrity of meaning, introductory sentence, details, use of keywords and main 
idea. The number of successful students in the criteria of paper layout (f= 1), use of tense 
suffixes (f= 1), grammar, punctuation, spelling errors (f= 2), plot (f= 2) and direct quotation 
or imitation (f= 2) is quite limited. 

It is seen that 21 students are successful in the criterion of putting a new title in the 
summaries written by the students in the second week and there are no summaries evaluated 
as inadequate in the related criterion. In the criterion of the main idea, in which there are no 
successful students in the first week, 7 students are successful. The number of summaries 
evaluated as inadequate in the introductory sentence (f= 15) and the use of tense suffixes (f= 
12) is high. There is an improvement in all of the criteria compared to the first week, however, 
the number of successful students in the criteria of the integrity of meaning (f= 1), plot (f= 1), 
introductory sentence (f= 1), use of time affixes (f= 2), direct quotation or imitation (f= 2) is 
quite low. 

In the summaries of the third week, it is seen that all of the students (f= 22) put a new 
title to their summaries. There is no successful student in the introductory sentence criterion. 
The number of successful students in the use of keywords (f= 1) and main idea (f= 1) criteria 
is quite low. While | student was successful in the main idea criterion in the third week, 11 
students were successful in the fourth week. 

In the fourth week, the percentage of success (100 %) is repeated in the criterion of 
putting a new title. The lowest number of successful students is found in the criterion of integrity 
of meaning (f= 5). In all other criteria, the number of successful students is improved. In the 
introductory sentence criterion, where there were no successful students in the third week, there 
are 13 successful students in the fourth week. While one student was successful in the main idea 
criterion in the third week, this number is eleven in the fourth week. 

In the fifth week summaries, the number of insufficient students in the criteria of 
introductory sentence (f= 8), plot (f= 5), details (f= 4), keywords (f= 5) and main idea (f= 6) 
increased compared to the previous week. 

In the sixth week summaries, it is seen that the number of students who are successful 
in the criteria of putting a new title (f= 22), number of paragraphs (f= 21), main idea (f= 19) 
and introductory sentence (f= 18) is quite high. Although the level of success is at the top in all 
criteria as of the sixth week, there are students who are evaluated as inadequate in criteria other 
than the number of paragraphs and putting a new title. The criterion with the highest number of 
inadequate students (f= 5) is direct quotation or imitation. 

The weekly course of summarization achievement determined according to the average 
scores obtained from the rubric in which text summaries were evaluated is given in Figure 1. 
The first 3 items of the rubric are about the format properties of the summary, items 4-10 are 
about the content and the last two items are about the style. 
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Figure 1 
Weekly Development of Summarization Achievement 
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The results of the 6-week training period are shown in different colors. The dark blue 
line in Figure | represents the lst week of the pretest scores and the green line represents 
the 6th week of the posttest scores. In the first week of the study, the scores obtained from 
summarization the text with an appropriate number of paragraphs (X = 2.36, SD = 0.73) and 
putting a new title to the summary text (¥ = 2.27, SD = 0.70) are above 2. The scores obtained 
from other criteria are below 2. In the Ist week, the lowest scores are obtained for starting the 
summary text with an introductory sentence that reflects the topic (Y = 1.05 SD = 0.21) and 
ending the text with a sentence that gives the main idea (X¥ = 1.05, SD = 0.21). In the second 
week, improvement is observed in all items. All format features of the abstract are above 2 
points. Items 4-10 in the content features of the text and items 11 and 12 in the stylistic features 
continue to remain below 2 points. In the third week, the scores of the criteria related to form 
features continue to increase and are above 2 points. The score obtained from the criterion of 
putting a new title to the summary text (X = 3, SD = 0.00) is at the highest level. In the fourth 
week, the scores of all criteria except the criterion of compliance with grammar, spelling and 
punctuation rules (¥ = 1.95, SD = 0.79) are above 2. Although all of the scores are above 2 in 
the fifth week, it is noteworthy that there is a decrease in the criteria of paper layout (-0.14), 
number of paragraphs (-0.14), putting a new title (-0.05), introductory sentence (-0.0909), use 
of keywords (-0.05), plot (-0.09), and details (-0.14). In the 6th week, when the research was 
completed, and the post-test data were obtained, an increase was realized in all criteria. 

The averages of the students' summarization achievement during the 6-week training 
period were examined and the data are presented in Table 4. 
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Table 4 
Weekly Averages of Summarization Achievement 


Week N X SD Min. Max. 
1 22 1.52 30 1.08 2.17 
2 22 1.90 42 1.25 2.83 
3 22 1.94 43 1.33 2.92 
4 22 2.31 50 1.17 3.00 
5 22 2.30 AQ 1.42 3.00 
6 22 2.61 40 1.33 3.00 


As seen in Table 4, the lowest mean is at week 1 (X = 1.52, SD = 0.30) and the highest 
mean is at week 6 (X= 2.61, SD = 0.40). In weeks 1, 2 and 3, the mean of achievement is below 
2. In weeks 4, 5 and 6, the achievement average is above 2. While the mean achievement of 
the summaries increases during the first 4 weeks, the mean achievement of the summaries in 
the 5th week (¥ = 2.30, SD = 0.49) decreases (-0.01) compared to the previous week (X = 2.31, 
SD = 0.50). When the averages of summarization achievements between weeks are examined, 
the highest increase (+0.38) is between the Ist and 2nd week. 


Results Related to the Effect of Teaching Macrostructure and Superstructure on 
Summarization Narrative Texts Achievement 


The Wilcoxon signed-rank test was conducted to explore the effect of teaching 
macrostructure and superstructure on summarization narrative texts achievement, and the 
results are presented in Table 5. 


Table 5 
Pretest-Posttest Difference in Summarization Achievement 


Posttest - Pretest N Mean Rank Sum of Ranks Z p 
Negative Ranks 1 1.00 1.00 -3.97 .001 
Positive Ranks 20 11.50 230.00 

No difference 1 


As seen in Table 5, in the post-test, one student's achievement is lower than in the pre- 
test, and one student's achievement is the same. In the post-test, 20 students increase their 
achievement. When the students' summarization achievement was compared between the 
posttest and pretest, it was determined that summarization achievement showed a significant 
difference in favor of the posttest (Z = -3.97, p<.01). According to this result, teaching 
macrostructure and superstructure is effective in summarization narrative texts achievement. 


Discussion 
In this research, it was explored how strategies based on determining the macrostructure 
and superstructure characteristics of the text affect the achievement of text summarization. As 


a result of the research conducted for 6 weeks in a single group pretest and posttest design, text 
summarization achievement showed significant differences in favor of the posttest. 
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It is noteworthy that in the first week of summarization achievement, there were no 
students who were successful in the criteria of semantic integrity, introductory sentence, 
details, use of keywords and main idea. Except for the criteria of adding a new title (f= 9) 
and number of paragraphs (f= 11), the number of successful students remained limited in 
other criteria. These results show that students' summarization skills were weak before the 
summarization strategies training. The reason for this may be that summarization skills are tried 
to be taught only through activities in course books. In the studies of Sengiil (2005), Dilidtizgiin 
(2013), Karadag (2019), Ozdemir and Eroglu (2022), it was determined that the summarization 
activities in Turkish course books were insufficient for teaching summarization, indicating that 
there may be problems in developing summarization skills through activities. 

According to the results obtained from the evaluation of summarization achievement, the 
highest increase in summarization scores (+0.3826) was between the Ist and 2nd weeks. This 
result is related to the second week of summarization teaching. However, the lowest increase 
in summarization achievement scores (+0.0417) occurred between the 2nd and 3rd weeks. This 
finding shows that the rate of development of summarization achievement does not follow 
a regular course and it is important to repeat the applications to reach the desired level of 
development. 

The students who were evaluated as "successful" in summarization achievement, semantic 
integrity and use of tense suffixes increased every week. However, students who were evaluated 
as inadequate in the semantic integrity criterion remained stable in the first three weeks. Semantic 
integrity is a criterion based on the solid establishment of grammatical connections and the use 
of appropriate transition and connection expressions. Data from the first three weeks show that 
students had difficulty providing transition and connection points in their summary texts. In the 
study conducted by Deneme (2008) with secondary school students, it was determined that the 
students had problems in providing semantic integrity in their summaries. 

It was determined that although the students’ achievement in summarization in the six- 
week training period increased in the use of details and keywords, they could not reach the 
desired level. Problems with deleting details and using keywords show that students have 
difficulty distinguishing important information from unimportant information. In different 
summarization studies conducted with secondary school students, it was determined that there 
were problems in the strategies of deleting unnecessary information and separating important 
information from unimportant information (Aktas & Bayram, 2017; Aydin, 2022; Bahap Kudret 
& Baydik, 2016; Cikrike1, 2008; Erdem, 2012; Karatay & Okur, 2012; Ozcakmak, 2015). 
Deletion is the most fundamental strategy in macrostructure construction rules (Brown & Day, 
1983). The development of this strategy requires a long-term process. 

In the results of the posttest of summarization achievement, it is seen that the least 
success was in the direct quotation and imitation criteria. In this criterion, 5 students were 
evaluated as inadequate, 5 students needed improvement and 12 students were evaluated as 
successful. The fact that students transfer information from the source text to their summary 
texts through direct quotation and imitation shows that they have problems in their ability to 
express it in their own words. In the study conducted by Cikrik¢i (2008), it was determined 
that secondary school students frequently included direct quotations in their summary texts. 
Along with the direct quotation and imitation criteria, the criterion with the lowest number 
of successful students in the posttest was grammar, punctuation, and spelling errors. In the 
summaries examined, it was observed that students made especially the use of capital letters, 
letter errors and missing punctuation. The findings coincide with the findings of Sigimlii's 
(2020) study, which examined the application levels of secondary school students' spelling and 
punctuation rules. Grammar, spelling, and punctuation include the basic rules of Turkish. In 
order to eliminate the relevant deficiencies, students can internalize the relevant rules by using 
them throughout their lives. 


https://doi.org/10.33225/pec/24.82.85 ISSN 1822-7864 (Print) ISSN 2538-7111 (Online) 


Ebubekir EROGLU, Serpil OZDEMIR. The effect of macrostructure and superstructure teaching on summarization achievement 


When the weekly averages of summarization achievement were examined, a regular 
increase was observed in the first four weeks, while a decrease was observed in the fifth 
week averages. The text “Last Birds” was used in the fifth week summaries. The fact that 
the readability level of the text is medium difficulty may be the reason for the decrease in the 
average. Hellmann and Ehri (2020) determined in their study with fourth grade students that 
more successful summaries were obtained from easier texts. Also, this story, unlike others, is 
a situation story. Another reason for the decrease in averages may be that students are more 
accustomed to event stories. In the first three weeks of the research, the average summarization 
achievement scores were below 2. The average score remained below 2 in all 6 criteria in which 
the summary text was evaluated in terms of content. The content criteria of the summary are 
directly related to reading comprehension skills. It can be said that the average being below 
2 in content criteria during a three-week period is an indication that students have problems 
in understanding and transferring what they read. In Ozdemir's (2018) study, it was observed 
that the summarization achievement of Turkish teacher candidates reached an average of over 
2 points in all criteria in the 3rd week. The secondary school students in this study increased 
over 2 points in all criteria in the 5th week. This result shows that the age factor is effective in 
the development of summarization achievement. As a matter of fact, in the study conducted 
by Brown et al., (1983) with student groups of different ages, it was determined that the age 
factor was effective in summarization achievement. The average, which rose above 2 by the 
fourth week, rose above 2.5 in the 6th week of the research. At the end of the sixth week, the 
highest average was recorded in all criteria. Considering the success averages obtained in the 
research and the results of other studies, it can be said that it is important to make planning 
in connection with the development characteristics of student groups when determining the 
duration of summarization strategies training. 


Conclusions and Implications 


In this study, the effect ofsummarization teaching based on determining the macrostructure 
and superstructure features of the text on the achievement of narrative text summarization was 
investigated. As a result of the research, a significant difference in summarization achievement 
was seen in the pre-test and post-test averages in favor of the post-test. An evaluation was made 
with 12 criteria regarding form, content, and style. Achievement has increased on all of these 
criteria. The least increase in criteria was seen in grammar, spelling, punctuation, and direct 
quotation and imitation criteria. The rate of successful students in these criteria was 54.5%. 
59% of students were successful in details and keyword usage, and at least 68% were successful 
in other criteria. However, it was observed that students had difficulty when switching from 
an easy level to a medium difficulty level and from an event story to a situation story. These 
conclusions provide valuable implications for both educational practice and further research. 

In order to improve students' summarization achievement, reading education should 
not be carried out only with questions about the meaning of the text. Teaching should be 
done on the elements that make up the macrostructure of the text and how these elements are 
arranged in the text according to their superstructure characteristics. The elements that make 
up the macrostructure of the text are the title, topic sentence, keywords, main and supporting 
ideas/events, content scheme, topic change markers and conclusion sentence. Depending on 
the superstructure of the text, these elements can be found in different parts of the text and 
different linguistic markers can be used. Analyzing the text in terms of these elements will 
contribute to the understanding of the text and is also important in determining which elements 
will be included in the summary. The research has shown that students need to study the texts 
repeatedly to determine the topic sentence, keywords, plot, and main idea and include them in 
the summary. It has been observed that these studies should continue for at least 4 weeks with 
7th grade secondary school students. 
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Based on the research results, it is recommended that text structure criteria be included in 
the main language teaching program and repeated with different texts for a long time. Teachers 
should guide students in the summarization process and structure summarization education 
gradually. During the summarization process, they should clearly tell students what they need 
to do and be a model in showing them how to do it. Providing feedback in noticing the mistakes 
or truths made during the summarization process positively affects the students' achievement. 
Therefore, teacher and peer feedback should be offered. 

This research is limited to narrative texts and secondary school students. There may 
be differences in determining summarization achievement depending on text types and the 
student's level. In further research, whether the achievement of summarization narrative and 
informative texts differs can be examined based on text structure teaching. Determining how 
long it takes for different age groups to develop narrative text summarization achievement is 
also one of the issues that need to be investigated. 

The readability level of the texts and the organization of their content have an impact on 
the achievement of summarization. In summarization research, a transition should be made from 
easy texts to difficult texts. In the selected texts, the content should be presented in accordance 
with the genre characteristics. 
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